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AUTOMATED  MAINTENANCE  SYSTEM  TEST  PROGRAM 
INCREMENT  VI  "PRODUCTION  SCHEDULING" 
EVALUATION  REPORT 


ABSTRACT 

The  purpose  of  this  report  is  to  document  evaluation  results  of  automated 
production  scheduling  processes  within  the  Automated  Maintenance  System  (AMS). 
The  processes  were  developed  and  tested  at  Dover  AFB,  Delaware.  Maintenance 
management  functions  affected  by  AMS  Increment  includes  scheduling  and 
tracking  of  reparable  parts  and  items  requiring  calibration.  Workcenters 
affected  at  the  time  of  the  appraisal  included  all  Avionics  Maintenance  shops, 
the  Avionics  Reparable  Processing  Center  (RPC),  and  the  Field  Maintenance 
Environmental  Systems  Shop.  This  report  contains  the  evaluation  methodology, 
findings  and  recommendations  for  future  program  development. 
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EXECUTIVE  SUMMARY 


The  purpose  of  Che  Automated  Maintenance  System  (AMS)  Test  program  is  to 
measure  the  effects  of  automating  aircraft  maintenance  management  processes  in 
a  live  base  environment. 

The  system  consists  of  the  on-board  C-5  aircraft  Malfunction,  Analysis, 
Detection  and  Recording  System  (MADARS)  which  collects  in-flight  systems  data, 
and  the  C-5  Ground  Processing  System  (GPS)  which  is  the  maintenance  data 
processor.  GPS  was  expanded  to  process  a  wide  variety  of  maintenance 
information.  The  expanded  GPS  is  known  as  AMS  or  AMS/GPS.  AMS  was  developed 
incrementally  with  six  increments  completed  to  date.  Evaluation  results  of 
Increments  l  through  V  were  documented  in  June  1981.  This  report  covers 
increment  VI,  "Production  Scheduling."  Increment  VI  provides  a  tracking 
system  for  "off-equipment"  actions  including  test,  calibration,  and  repair  of 
components  in  the  shop  environment.  It  is  intended  to  increase  the  speed  and 
accuracy  of  in-shop  maintenance  processes  by  automating  or  eliminating  a 
number  of  manual  forms,  computing  job  standards  and  shop  backlogs,  and 
replacing  tracking  boards  used  in  the  scheduling  process. 

Our  evaluation  approach  was  to  determine  whether  program  objectives  and 
postulated  benefits  listed  in  the  Functional  Description  (FD)  were  realized. 
We  also  looked  at  supply  repair  cycle  data  to  see  if  parts  are  moving  through 
the  system  faster.  The  Air  Force  Human  Resources  Laboratory  evaluated  human 
factors. 

At  the  time  of  the  evaluation,  the  Avionics  Maintenance  Squadron  Bhops 
were  using  Increment  VI.  Field  Maintenance  Squadron  was  operating  under  the 
manual  scheduling  system,  with  the  exception  of  the  Environmental  Systems 
Shop.  In  comparing  manual  versus  automated  processes,  we  used  Avionics 
Maintenance  workcenters  as  the  test  group,  while  Field  Maintenance  workcenters 
served  as  the  control  group. 

Wine  of  the  program  objectives  were  fully  met  and  three  were  partially 
met.  Many  postulated  benefits  were  realized  to  varying  degrees.  The  most 
notable  benefit  was  the  savings  in  time  required  to  update  Precision 
Measurement  Equipment  (PME)  records.  Supply  data  showed  no  significant  change 
in  average  repair  cycle  time  (RCT)  since  Increment  VI  was  implemented  (a 
reduction  in  RCT  was  expected  if  Increment  VI  had  improved  overall  repair 
cycle  management);  but  average  Not-Reparable-This-Station  (NRTS)  time  showed  a 
significant  increase  (again  a  reduction  was  expected).  However,  both  "block 
scheduling"  and  the  significant  policy  change  of  completing  maintenance  data 
input  prior  to  turn-in  to  supply  masked  the  potential  time  savings  of 
increment  VI.  Human  factors  assessment  showed  a  continuous  improvement  in 
users'  acceptance  of  the  system. 

increment  VI  provided  a  real-time  tracking  system.  However,  use  of  the 
"block  scheduling"  method  of  processing  reparables  limited  its  effectiveness. 
Keparables,  with  few  exceptions,  were  being  processed  in  mass  twice  each  day. 
Current  reparable  processing  procedures  should  be  reviewed  by  appropriate 
management  and  revised  to  take  full  advantage  of  the  real-time  capabilities  of 
Increment  VI. 


Overall,  AMS  Increment  VI  has  established  a  complete  reparable  processing 
and  PME  database.  However,  users  expressed  frustration  over  the  requirement 
to  make  duplicate  Inputs  to  both  the  Increment  VI  and  supply  computer  systems. 
This  unnecessary  redundancy  could  and  should  be  eliminated  through  a  systems 
Interface  which  is  planned  under  AMS  Increment  VII.  This  interface  should  be 
developed  as  soon  as  possible  to  fully  use  the  real-time  reparable/PME 
database  within  Increment  VI. 
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CHAPTER  1  -  INTRODUCTION 


1.  The  purpose  of  the  Automated  Maintenance  System  Test  program  is  to  treasure 
the  effects  of  automating  aircraft  maintenance  management  processes  in  a  live 
base  environment.  AMS  Increments  I  through  V  were  evaluated  in  June  1981. 
This  report  covers  Increment  VI,  "Production  Scheduling"  processes. 

<!.  Background: 

a.  The  C-5  aircraft  has  an  onboard  Malfunction,  Analysis,  Detection  and 
Recording  System  (MADARS)  which  is  processed  by  a  Ground  Processing  System 
(GPS)  at  the  three  C-5  sites.  The  C-5  MADARS/GPS  consists  of  an  IBM  1130  at 
each  C-5  base  and  a  Central  Data  Bank  (CDD  to  program  objectives,  postulated 
benefits,  repr.ir  cycle  system  impacts,  and  human  factors. 

b.  In  February  1975,  MAC  submitted  a  Data  Automation  Requirement  (DAR) 
recommending  standard  Air  Force  application  of  the  C-5  GPS.  In  response  to 
the  MAC  DAR,  three  Air  Staff-directed  conferences  were  held.  These 
conferences  resulted  in  identification  of  several  maintenance  management 
processes  to  be  tested.  The  processes  were  separated  into  seven  increments 
identified  below: 

I  WORK  ORDER  GENERATION/CLOSE  OUT 

II  DEBRIEFING 

III  PERSONNEL  AVAILABILITY 

IV  JOB  FOLLOWING 

V  DESIGNED  OPERATIONAL  CAPABILITY 

VI  PRODUCTION  SCHEDULING 

VII  MaINTENANCE/SUPPLY  INTERFACE 

3.  Scope:  This  evaluation  was  conducted  to  determine  if  AMS/GPS  processes 
tor  Increment  VI  met  the  stated  program  objectives  and  postulated  benefits  in 
the  functional  description.  The  program's  effects  on  repair  cycle  time  were 
also  measured  and  a  human  factors  study  was  conducted  to  determine  users' 
acceptance. 
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CHAPTER  2  -  METHODOLOGY 


1.  Hypothesis:  The  overall  hypothesis  for  the  AMS  Test  Program  is  that  the 
"process  will  improve  management  procedures,  allowing  more  time  for 
maintenance  supervisors  to  deal  with  oroduction  oriented  requirements. 
The  specific  hypothesis  for  Increment  VI  is  that  "this  process  will  save 
manhours  by  eliminating  manual  procedures  and  files,  improve  accuracy  of 
repair  cycle  data,  and  reduce  Due-In-From-Ma intenance  (U1FM)  reconciliation 
t  ime  .  "  2 

2.  Approach:  Four  areas  were  evaluated:  program  objectives,  postulated 

benefits,  effects  or.  repair  cycle  time,  and  human  factors.  In  comparing  the 
manual  processes  to  the  automated  processes,  we  used  the  Field  Maintenance 
Squadron,  which  did  not  have  Increment  VI,  as  the  control  group,  and  the 
Avionics  Maintenance  Squadron,  which  did  have  Increment  VI  as  the  test  group, 
we  carefully  excluded  data  from  dissimilar  procedures  such  as  multiple-shop- 
processed  items  in  FPS  to  insure  valid  results. 

1.  Methodology: 

a.  The  program  objectives,  as  stated  in  the  Functional  Description  (FD), 
were  evaluated  to  determine  if  they  had  been  fully  met,  partially  met,  or  not 
met.  Each  objective  was  broken  into  key  elements  whenever  possible.  The 
presence  or  absence  of  these  elements  determined  findings.  The  evaluation  was 
conducted  through  direct  observation  of  system  use,  stopwatch  tests,  and 
comparison  of  data  samples  from  maintenance  and  supply  computer  products  and 
input  source  documents. 

b.  The  postulated  benefits  assessment  was  conducted  to  determine  if  each 
benefit  was  realized  and  the  value  of  the  benefit.  Evaluation  processes 
included  direct  observation  of  system  use,  stopwatch  studies,  user  Interviews, 
and  quantitative  analysis  of  supply  and  maintenance  data. 

c.  .Supply  repair  cycle  data  was  analyzed  to  determine  if  the  system  had 

impacted  Repair  Cycle  Time  (RCT)  and/or  Not-Reparable-This-Station  (NETS) 

processing  time.  This  data  was  collected  from  Dover  AFb  supply  computer 
tapes.  The  test  hypotheses  were: 

(1)  Average  Repair  Cycle  Days  -  Increased  efficiency  in  parts 

processing  should  be  reflected  in  a  reduction  of  RCT. 

(2)  Average  NKTH/Condemned  Days  -  increased  efficiency  in  parts 

processing  should  be  retlected  in  a  reduction  of  NRTS/Condemned  time. 

d.  Human  Factors  were  assessed  by  the  Air  Force  Human  Resources 

Laboratory  through  questionnaires  and  personal  interviews.  The  areas  measured 
were  training  required,  implementation  difficulty,  process  usability,  and  user 


CHAPTER  3  -  EVALUATION  RESULTS 


This  chapter  contains  analyses  and  findings  related  to  program  objectives, 
postulated  benefits,  repair  cycle  system  impacts,  and  human  factors. 


SECTION  ONE  -  PROGRAM  OBJECTIVES  EVALUATION 

OBJECTIVE  1:  Provide  schedulers  with  a  single,  integrated  data  base  to  store 
all  UIFM  and  Precision  Measurement  Equipment  (PME)  asset  status  information. 

This  objective  was  broken  into  four  elements:  (1)  single  database,  (2) 

integrated  database,  (3)  complete  DIFM  status,  and  (4)  complete  PME  status. 

ANALYSIS: 

(1)  Single  data  base  -  TO  00-20-3  requirements  for  DIFM  information 

was  contained  within  the  AMS/GPS  single  data  base  as  was  PME  asset 
information.  AFM  67-1  data  requirements  were  not  contained  in  a  single 

database  but  are  maintained  separately  in  the  Supply  computer.  This 

necessitated  updating  of  supply  data  through  manually  generated  forms  which 
often  contained  information  redundant  to  AMS/GPS  information. 

(2)  Integrated  data  base  -  The  term  "Integrated"  as  applied  in  the  FD 

refers  to  the  system's  ability  to  crossfeed  information  to  eliminate  redundant 

inputs.  From  the  maintenance  standpoint,  the  data  base  is  fully  integrated 

with  the  exception  that  AFTO  Form  350  and  AFTO  Form  349  data  is  not  crossfed. 

(3)  UIFM  status  information  -  Complete  DIFM  status  was  available  in 

AMS/GPS . 

(4j  PME  asset  information  -  The  Master  ID  Listing  was  fully  automated 
and  "real-time"  under  Increment  VI.  Calibration  due  dates  were  automatically 
updated  by  MDC  inputs,  eliminating  the  use  of  the  AF  Form  1530  in  updating  the 
Master  ID  Listing. 

FINDING:  PARTlaLLY  MET.  The  ability  to  crossfeed  AFTO  Forms  349  and  350 

uata,  and  the  ability  to  crossfeed  data  between  maintenance  and  supply 
information  systems  were  the  limiting  factors. 

UBJECTIVE  2:  Provide  on-Line  inquiry  capability  to  permit  monitoring  of 
individual  assets  status  and  location. 

aNALYSlS:  Asset  status  and  location  (including  ETJC)  was  available  on-line. 

However,  the  capability  was  limited  to  listing  all  parts  in  a  particular 
workcenter  or  individual  parts  by  AFTO  Form  350  tag  number.  The  system  could 
not  be  used  to  find  the  AFTO  Form  350  when  a  part's  paperwork  was  lost  or 
unreadable.  The  system  could  not  be  used  to  determine  the  status  and  location 
of  a  grouping  of  parts  such  as  by  national  stock  number,  manufacturer's  part 
number,  or  by  work  unit  code.  Further,  users  had  to  "page  through"  an  entire 
shop  report  to  locate  an  individual  part  if  the  tag  number  was  unknown.  A 
program  to  locate  parts  and  status  by  national  stock  number,  manufacturers 


part  number,  and/or  work  unit  code  was  needed  to  allow  individual  asset 
monitoring. 

EINDINC:  FULLY  MKT.  The  objective  was  partially  met  at  the  time  of  the 
evaluation.  Following  the  evaluation,  AMS/GPS  programmers  have  added  the 

capabilities  to  terminate  the  shop  report  and  to  locate  parts  by  a  variety  of 
categories . 

OBJECTIVE  3:  Provide  asset  inventory  control  through  automated  records 

creation  when  an  asset  enters  the  repair/calibration  cycle. 

This  objective  was  broken  into  two  elements:  (1)  Asset  inventory  control, 
and  (2)  Automated  records  creation  for  assets  undergoing  repair/calibration. 

ANALYSLS: 

(1)  asset  Inventory  Control  -  Comparisons  of  actual  asset  status  and 
Location  versus  recorded  status  and  location  revealed  no  discrepancies.  A 
high  degree  of  asset  control  was  readily  apparent. 

(2)  Automated  records  creation  -  All  assets  entering  shops  with 
increment  VI  capability  were  processed  through  RPC  using  automated  records. 

FINDING:  FULLY  MET.  Data  contained  in  Increment  VI  programs  meets  and 

exceeds  the  T.u.  UU-20-3  production  control  requirements. 

OBJECTIVE  4:  Provide  an  inquiry  system  for  obtaining  summary  labor  hour 

backlog  data  for  off-equipment  work  (Automated  AF  Form  2443)  and  PME  assets 
awaiting  repair/calibration. 

ANALYSIS:  The  system  automatically  computed  and  displayed  current  shop 

backlog  data  in  under  ten  seconds  in  the  Avionics  RPC  which  had  Increment  VI. 
Ln  the  FMs  RPC,  which  did  not  have  Increment  VI,  the  scheduler  required  more 
than  a  halt  hour  to  complete  backlog  calculations  and  fill  out  the  AF  Form 
2443.  The  automated  AF  Form  2443  was  easier  to  read  and  understand,  and 
contained  the  same  information  as  the  manual  form.  Also,  the  automated  AF 
Form  2443  contained  real-time  information  while  the  manual  form  represented 
"snap-shot"  information. 

FINDING:  FULLY  MET.  A  direct  manhour  savings  of  more  than  a  half  hour  each 

workday  was  realized  as  a  result  of  this  program. 

OBJECTIVE  3:  Provide  a  suspense  system  to  monitor  estimated  times  of  job 
completions  (ETJCs). 

ANALYSIS:  This  objective  was  broken  into  two  elements:  (1)  ETJC  suspense 

system  and  (2)  ETJC  monitoring. 

(.1  )  ETJC  suspense  system  -  Avionics  RPC  schedulers  demonstrated  that 
ETJCs  were  present  in  the  system,  but  In  many  cases,  no  ETJC  had  been 
established  lor  individual  items  in  the  in-work  (1NW)  category.  No  overdue 
ETJCs  were  noted  during  the  visit,  but  in  follow-up  checks  Irom  our  AMS/CPS 
terminal  at  Cunter  AFB,  we  found  many  overdue  ETJCs.  The  system  does  not 
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force  an  ETJC  input  nor  does  it  automatically  notify  RPC  or  the  shop  when  an 
c-TJC  goes  overdue. 

(2)  ETJC  monitoring  -  The  system  provided  the  means  to  monitor  ETJCs , 
although  the  program  used  for  this  purpose  was  not  very  convenient  since  the 
user  had  to  go  through  the  entire  shop  report  to  find  an  individual  ETJC . 
Avionics  RPC  schedulers  said  that  individual  ETJCs  were  checked  twice  daily 
except  for  certain  critical  items  that  were  checked  hourly. 

FLNDINC:  PARTIALLY  MET.  At  the  time  of  the  evaluation,  the  program  lacked 

the  capability  to  notify  schedulers  that  an  ETJC  was  overdue.  AMS/GPS 
programmers  have  since  added  a  program  to  quickly  display  a  list  of  next  due 
ETJCs  for  all  shops,  but  the  system  still  has  no  automatic  ETJC  due/overdue 
notification  program. 

OBJECTIVE  6:  Provide  automatic  update  of  PME  records  from  MDC  close-out  data 
entered  in  the  system. 

ANALYSIS:  The  PME  scheduler  demonstrated  automatic  update  of  PME  data  through 
MDC  cLose-out  along  with  a  programmatically  updated  Master  ID  Listing.  The 
scheduler  said  that  before  Increment  VI  was  implemented,  it  took  her  two  full 
weeks  out  of  each  month  to  update  this  listing.  This  does  not  include  the 
time  spent  by  the  owning  workcenters  in  keeping  their  listings  up  to  date. 
v»orkcenters  periodically  receive  notification  of  items  coming  due  calibration 
in  the  near  future.  This  feature  has  virtually  eliminated  items  overdue 
cal i brat  ion . 

EIND1NC:  FULLY  MET.  In  our  opinion,  this  is  the  most  outstanding  system 

tea tore  tor  Increment  VI. 

UBJECTIVE  7:  Provide  accurate  asset  status  data  to  enhance  the  Mission 
Capability  System  (MICAP)  verification  process. 

inis  objective  was  broken  into  two  elements:  (1)  Accurate  asset  status 

Data,  and  ( / )  enhancement  of  MICAP  verification. 


.•vNalYS  Is : 


(1)  Accuracy  of  asset  status  data  was  verified  by  comparing  actual 
status  .and  location  of  parts  with  the  information  shown  in  the  AMS/GPS 
computer.  No  discrepancies  were  noted. 

(2)  Enhancement  of  MlCAP  verification  -  MICAP  verifiers  stated  that 
their  primary  method  of  checking  parts  status  and  location  was  to  call  RPC. 
Ihe  computer  was  used  only  as  a  last  resort.  They  indicated  they  would  use 
the  computer  only  if  they  were  able  to  call  up  assets  by  NSN,  part  number,  or 

Wilt  . 

FINDING:  FULL  MET.  This  objective  was  not  met  at  the  time  of  our  visit. 

/VMS /GPS  programme rs  have  since  added  a  program  to  call  up  parts  status  and 
location  by  NSN  in  a  variety  ot  categories. 

OBJECTIVE  6:  Provide  job  average  for  component  repai r/cali brat  ion . 


Subprocess  C  -  Ordering  repair  parts. 

Act  tvi  ties  _ Average  Time 


1  AMS  Fill  in  AF  2413  3  min  30  sec 

1  FMS  Fill  in  AF  2413  3  min  30  sec 


AMS  input  AFTO  330  status  change  (AWP)  1  min  52  sec 

2  F MS  Annotate  AFTu  350  part  1  (AWP  status)  OS  sec 


3  AMS  Attach  doc.  to  part  and  send  to  AWP  30  sec 

3  FMS  Attach  doc.  to  part  and  store  in  AWP  area  30  sec 


Total  Process  Time  -  AMS  (Inc  VI  Shops)  5  min  52  sec 

lot  il  Process  Time  -  FMS  (Non-Inc  VI  Shops)  4  min  08  sec 


eut*pr  >cess  L)  -  Close  out  ot  NRTS  action  in  the  shop. 

\cr  tv  i  ties  _ Average  Time 


1  AMs  No  Action  Required  00  sec 

1  h MS  Annotate  Shop  Log  42  sec 


..  tMS 

Complete  DU  1577-2  unserviceable  (2  cy ) 

2  mi  n 

02 

sec 

.  FMs 

Complete  L)U  1  577-2  unserviceable  (2  cy ) 

2  min 

02 

s  ec 

<  AMS 

Input  AFTO  349  Data 

2  min 

57 

sec 

3  FMS 

Input  AFTu  349  Data 

2  mi  n 

57 

sec 

-t  AM.- 

RPC  notified  by  9129  program 

00 

sec 

-  f  MS 

Notify  RPC  when  work  completed 

1  mi  n 

00 

sec 

»  AMS 

Complete  Automated  AFTO  350 

10 

sec 

)  i:  lh 

Complete  AFTO  Form  3  SO 

44 

sec 

»  AMs 
it  FMS 


Attach  documentation  to  part  for  RPC  processing 
Attach  documentation  to  part  for  RPC  processing 


35  sec 
35  sec 


SHOP  PROCESSING 


Subprocess  B  -  Processing  parts  for  repair  in  the  shop 

Act ivi ties 

• 

Average  Time 

1  AMS 

Enter  AKTO  Form  350  data  in  computer  (9128  screen) 

Printout  AKTO  Form  350,  attach  to  part  for  RPC 

proc. 

4 

min 

18 

sec 

1  KMS 

Fill  out  AKTu  Form  350  tag,  attach  to  part  for 

RPC  proc 

2 

min 

13 

sec 

2  AMS 

No  action  required 

00 

sec 

2  PMS 

Annotate  shop  reparable  log  (part  to  RPC  for  proc) 

42 

s  ec 

3  AMS 

No  action  required 

00 

sec 

i  KMS 

Annotate  shop  reparable  log  (part  rec'd  from  RPC) 

42 

sec 

4  AMS 

Enter  AKTO  349  data 

3 

min 

00 

sec 

4  FMS 

Enter  AKTO  349  data 

3 

min 

00 

sec 

RPC  notified  by  9129  program  report 

00 

sec 

m 

Notify  RPC  of  repair  completion  by  phone 

1 

min 

00 

sec 

0  AMS 

Complete  DD1574  serviceable  tag 

2 

min 

02 

sec 

1)  KMS 

Complete  UD1574  serviceable  tag 

2 

min 

02 

sec 

7  AMS 

Complete  Automated  AKTO  350 

10 

sec 

7  KMS 

Complete  AKTO  form  350  (work,  completed) 

30 

sec 

8  AMS 

No  Action  Required 

00 

sec 

0  KMS 

Annotate  shop  reparable  log  (completed) 

42 

sec 

4  AMS 

No  action  required 

00 

sec 

4  KMS 

Annotate  l<-2b  report 

30 

sec 

1  o  t  a  1 

Process  time  -  AMS  (Inc  VI  Shops) 

9 

mi  n 

30 

sec 

1  o  Lai 

Process  time  -  KMS  (Non-Inc  VI  Shops) 

11 

mi  n 

2  1 

sec 
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APPENDIX  A  -  EVALUATION  WORKSHEETS 


WORKSHEET  1 

a.  Evaluation  Objectives:  Determine  if  AMS  Increment  VI  has  provided  a 
measurable  manhour  savings  in  the  time  required  to  process  parts  for  repair  in 
RPC  and/or  the  shops.  Als^  determine  if  manual  documentation  has  been  reduced 
and  if  D1FM  reconciliation  time  has  been  reduced. 

b.  Methodology:  Interview  RPC  personnel  and  shop  reparable  monitors  to 
determine  the  activities  involved  in  processing  reparables.  Perform  stopwatch 
checks  to  find  the  time  required  for  each  activity.  Compare  the  steps  and 
times  for  AMS  schedulers  (with  Increment  VI)  and  FMS  schedulers  (without 
Increment  VI). 


INITIAL  RPC  PROCESSING 


Subprocess  A  -  Processing  parts  from  RPC  to  the  shops. 

Act ivi t ies 

Average  Time 

1  AMS 

Inspect  documentation  attached  to  part 

03 

sec 

1  FMS 

Inspect  documentation  attached  to  part 

03 

sec 

i  AMS 

Enter  AFTO  Form  350  sched.  data  in  computer  and  print 

15 

sec 

2  FMS 

Fill  in  scheduling  data  on  AFTO  Form  350  and  stamp 

35 

sec 

3  AMS 

No  action  required 

00 

sec 

}  FMS 

Annotate  RPC  log 

04 

sec 

4  AMS 

"Visible  File"  automatically  generated 

00 

sec 

4  F’MS 

Place  350  part  2  in  visible  file  with  1348-1 

15 

sec 

3  AMS 

Attach  automated  AFTO  Form  350  to  part 

03 

sec 

3  FMS 

Attach  AFTO  Form  350  part  1  to  part 

03 

sec 

b  AMS 

Annotate  R-2b 

02 

sec 

o  FMS 

Annotate  R-26 

02 

sec 

7  AMS 

Kill  out  AK  Form  1998  (as  required) 

05 

sec 

/  FMS 

Fill  out  AF  Form  1998  (as  required) 

05 

sec 

lotal 

process  time  -  AMS  (Inc  VI  RPC) 

28 

sec 

total 

process  time  -  FMS  (Non-Inc  VI  RPC) 

1  mi  n  0  7 

sec 
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FOOTNOTES 


*Uata  Project  Plan  HAF-P76-11,  14  October  1981,  Annex  C, 

paragraph  3. 

^ibid.,  Annex  C,  Attachment  1,  paragraph  D. 

^Automated  Maintenance  Systems  Test  Program,  Prototype 
Description,  Increment  VI,  31  July  1981,  Section  1,  paragraph  2.2. 

4Ibid.,  Section  2,  paragraph  2.4.1. 

■>TO  00-20-3,  Section  2,  "Production  Control,"  paragraph  2-4. 


Section  A, 


Functional 


1.  Core  Automated  Maintenance  System  (CAMS)  developers  should  review  and  make 
maximum  use  of  Increment  VI  programs.  (OPR:  DSDO/LG)) 

3.  functional  managers  should  review  and  revise  existing  reparable  processing 
procedures  (block  scheduling)  where  Increment  VI  provides  a  better  way  of 
doing  business.  (OPR:  HQ  MAC/LG,  OCR:  436  MAW/MA)) 

h.  CAMS  developers  should  periodically  review  Increment  VII  progress.  (OPR: 
USDO/CC) ) 


b.  AMS /GPS  and  CAMS  developers  should  review  every  program  prior  to 
implementation  to  make  sure  that  users  do  not  have  to  feed  the  same 
information  to  different  systems  or  subsystems.  (CAMS  OPR:  DSDO/CC);  (AMS 
OPR:  HQ  MAC/LG)) 

b.  A  formal  user  to  programmer  interface  plan  should  be  developed  for  AMS/GPS 
Increment  VI  and  VII  and  for  CAMS.  This  should  include  hands-on  during 
development  and  "fix"  or  “enhance"  following  initial  fielding  of  software. 
(OPR:  DSUO/CC;  AMS  OPR:  HQ  MAC/LG) 


lb 


CHAPTER  4  -  CONCLUSIONS  AND  RECOMMENDATIONS 


SECT  LON  ONE  -  CONCLUSIONS. 

1.  The  processes  constituting  AMS  Increment  VI  were  found  to  be  technically 
and  operationally  feasible.  The  development  of  Increment  VII,  "Maintenance' 
Supply  Interface,"  is  critical  to  maximizing  the  ultimate  value  of  Increment 
VI. 

1.  Processes  constituting  Increment  VI  are  applicable  to  Air  Force  aircraft 
maintenance  organizations,  although  the  application  programs  are  not  directly 
transf errable  to  base-level  computer  systems.  This  is  due  to  the  dissimi¬ 
larity  between  the  program  structure  for  the  central  data  base-orientation 
Automated  Maintenance  System  and  the  standard  base-level  computers. 

-1 .  Changes  in  existing  reparable  asset  management  procedures  may  be  necessary 
to  realize  the  full  potential  of  Increment  VI.  The  "hourly  scheduling"  method 
is  better  suited  to  the  real-time  properties  of  Increment  VI  than  the  "block 
scheduling"  method  currently  in  use  at  Dover  AFB.  The  use  of  shop  repair  logs 
are  not  considered  necessary  for  Increment  VI  users. 

**.  The  AMS/CPS  system  has  provided  an  ideal  laboratory  environment  for  the 
development  and  testing  of  maintenance  information  systems  concepts.  Efforts 
to  develop  and  test  AMS  Increment  VII  should  be  continued  to  provide  better 
understanding  and  validation  of  interface  relationships  between  maintenance 
and  supply  policies,  procedures,  concepts,  and  information  systems. 

'>.  A  key  "lesson  learned"  in  this  evaluation  is  that  an  interface  between 
maintenance  and  supply  data  systems  is  essential  to  make  the  best  use  of 
reparable  asset  scheduling  programs.  The  most  common  user  complaint  was  "we 
have  to  feed  the  same  information  to  two  systems."  Future  systems  developers 
should  make  sure  maintenance  and  supply  sub-systems  are  able  to  crossfeed 
information  BEFORE  implementation. 

b.  another  "lesson  learned"  during  this  evaluation  was  the  need  to  establish 
a  formal  user  to  programmer  feedback  system.  Many  of  the  AMS/GPS  programming 
I  ixes  which  enabled  us  to  conclude  individual  objectives  were  "fully  met," 
were  identified  during  our  evaluation  visit.  When  we  provided  feedback  to  the 
AMS/GPS  programmers  of  the  users'  needs,  they  were  "unaware"  of  the  problem. 
In  developing  any  large  data  automation  data  system,  there  needs  to  be 
interface  between  the  users  and  the  programmers  both  during  development  and 
alter  fielding.  Additionally,  plans  need  to  be  developed  to  provide 
programming  support  after  a  new  system  is  fielded.  This  support  should 
include  both  "getting  the  bugs  out"  and  "enhancing"  the  capability  of  the 
sot tware  package . 


BELT  LON  TWO  -  RECOMMENDATIONS 

1.  AMS/GPS  Increment  VII  should  be  developed  as  soon  as  possible.  (OPR:  HQ 
MAC/ LG) ) 
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Table  4.  Potential  Time  Savings 


Activi ty 


Monthly  Manhours  Freed 


KPC  and  Performing  Workcenters 

Parts  Processing 

Per  Unit  (2.51  Min)  x  Avg.  Units  Proc. 

Month  (2114) 

88 

hours 

Shop  Backlog  Computation 

32.5  Min.  Per  Day 

20 

hours 

PMEL  Master  I.D.  Update 

80 

hours 

Total  Per  Month 

188 

hours 

3 


SECTION  FOUR  -  HUMAN  FACTORS 


Initial  measurements  were  taken  in  November  1983,  and  a  follow-up  study 
was  completed  in  May  1984.  Definite  Improvements  in  attitudes  were  apparent 
after  users  had  time  to  learn  the  system  and  programmers  had  made  needed 
changes.  The  AFHRL  survey  and  report  are  found  in  Appendix  B.  Below  is  a 
synopsis  of  key  AFHRL  findings. 

1.  Training  -  In  November  1983,  many  users  indicated  training  was  one  of 
the  program's  biggest  problems.  Only  5  percent  said  training  was  a  problem  in 
the  May  1984  surveys.  Once  past  the  “learning  curve",  users  found  the  system 
very  friendly. 

2.  Difficulty  in  Implementation  -  Users  indicated  their  jobs  were  made 
easier  by  Increment  VI  in  the  May  84  survey,  with  positive  responses  10-20 
percent  higher  than  in  the  November  83  survey. 

3.  Usability  -  b3  percent  of  the  users  indicated  the  automated  system  is 
less  difficult  than  the  old  manual  system.  Responses  in  May  84  were  much  more 
positive  than  in  the  November  83  survey. 

4.  User  Acceptance  -  Responses  in  this  area  indicate  the  system  is  well 
accepted  by  the  users  and  it  provides  them  with  the  information  they  need  to 
do  their  jobs.  63  percent  indicated  Increment  VI  should  be  a  permanent  part  of 
HMS/GPS. 


btCTLON  FIVE  -  SUMMARY 

Increment  VI  provides  a  significant  improvement  in  tracking  reparables 
through  the  repair  process.  There  are  still  some  additional  actions  required 
to  obtain  the  full  benefits  of  the  system.  However,  by  automating  this 
process,  supervisors  and  technicians'  time  has  been  freed  to  allow  them  to 
accomplish  their  assigned  jobs:  leading  and  repairing.  Summarized  below  is  a 
table  of  the  potential  time  savings  which  should  accrue  from  Increment  VI. 


Table  3.  Average  RCT  and  NRTS  Days  per  Unit 


RCT  DAYS  RCT  DAYS  NRTS  DAYS  NRTS  DAYS 

_ Non-VI  Shops _ Inc  VI  Shops  Non-Inc  VI  Shops  Inc  VI  Shop 

Pre  Inc  VI  6.15  3.64  4.76  2.48 

Post  Inc  VI  5.67  3.61  4.40  4.14 


RCT  and  NRTS  averages  are  shown  in  Table  3  above.  The  first  and  third 
columns  provide  averages  for  those  shops  which  did  not  have  Increment  VI.  The 
second  and  fourth  columns  show  the  averages  for  shops  which  did  have  Increment 
VI.  The  first  row  shows  "before"  averages  and  the  second  row  shows  "after" 
averages. 


RCT  for  Increment  VI  shops  is  relatively  unchanged  while  NRTS  days 
increased  more  than  a  full  day.  We  would  expect  to  see  both  RCT  and  NRTS 
dropping  in  shops  which  have  a  better  way  to  process  parts.  This  was  not 
tound  in  our  results. 

However,  the  Increment  VI  shops  were  affected  by  a  significant  change  in 
policy  from  the  non-increment  VI  shops.  Under  Increment  VI,  parts  cannot  be 
turned  into  supply  until  Maintenance  Data  has  been  input  against  the  repair 
action.  In  the  case  of  NRTS  items,  prior  to  Increment  VI,  an  item  declared 
NRTS  was  returned  to  supply.  Maintenance  Data  was  input  separately  and,  often 
trailed  the  supply  turn-in. 

In  addition,  in  our  interviews  with  AMS  RPC  personnel,  (see  Appendix  A, 
worksheet  2),  they  said  that  parts  were  not  moving  through  the  system  faster 
even  though  their  processing  time  and  manual  documentation  had  been  reduced. 
They  gave  two  primary  reasons  for  this:  (1)  parts  spent  more  time  awaiting 
transport  than  in  administrative  processing;  (2)  shops  did  not  update  MDC  in  a 
timely  manner,  causing  some  parts  to  be  held  in  RPC  until  the  following  day. 
Our  analysis  appears  to  support  their  view.  We  believe  Increment  VI  has 
likely  reduced  processing  time  but  that  time  has  been  masked  by  two  factors: 
(1)  mandatory  MDC  update  prior  to  turn-in  to  Supply  and,  (2)  the  use  of  "Block 
Scheduling’’^  (a  batch  processing  method)  with  a  real-time  Increment  VI  system. 
Holding  parts  until  MDC  is  complete  is  a  significant  advantage  to  the  repair 
process.  Therefore  the  increase  in  repair  time  associated  with  this,  in  our 
estimation  less  than  one  day,  is  worth  the  delay.  Additionally,  by  using  an 
"Hourly  Scheduling’’^  method  instead  of  "Block  Scheduling"  the  advantage  of 
Increment  VI  real-time  processing  would  reduce  RCT  and  NRTS  time. 

I IND1NC:  Increment  VI  has  not  significantly  impacted  average  repair  cycle 

time.  Average  NRTS  days  has  shown  a  rise  since  Increment  VI  was  implemented. 
This  is  attributed  largely  to  the  use  of  a  real-time  system  to  batch-process 
parts  through  the  system,  and  the  delays  in  completing  MDC  inputs. 


BENEFIT  S:  Eliminate  the  time  required  for  production  control  personnel  to 
compute  shop  backlog. 

ANALYSIS:  In  stopwatch  studies,  FMS  schedulers  took  32.5  minutes  to  compute 

and  record  the  reparable  asset  backlog  on  the  AF  Form  2443  without  Increment 
VI  (see  Appendix  A,  Worksheet  3).  In  AMS  RPC,  Increment  VI  automatically 

computed  and  displayed  on  demand  the  shop  backlog  report.  AMS  schedulers 

reported  that  before  increment  VI,  they  normally  took  from  15  minutes  to  one 
and  one  halt  hours  to  compute  and  record  backlogs  each  day. 

UNDINGS:  The  shop  backlog  computation  has  been  eliminated  for  Increment 

Vl-equipped  shops  and  has  resulted  in  a  direct  manhour  savings  and  has 

eliminated  the  manual  AF  Form  2443. 

BENEFIT  b:  Reduce  MICAP  verification  time. 

ANALYSIS:  At  the  time  of  the  evaluation,  MICAP  verifiers  were  not  using 

Increment  VI  to  get  reparable  parts  status,  location  and  ETJC .  Instead,  from 
Increment  VI,  they  would  have  to  call  up  the  9129  report  and  page  through  all 
assets  in  a  given  shop  and  manually  extract  the  needed  data.  Verifiers 
indicated  they  would  use  Increment  VI  if  they  could  get  parts  status  and 
location  by  stock  number,  part  number,  and/or  work  unit  code  (WUC). 

1 INDING:  Increment  VI  did  not  result  in  a  reduction  of  MICAP  verification 

time  because  they  found  the  system  unusable  for  this  purpose.  However, 
aMS /GPS  programmers  have  since  developed  a  new  program  to  facilitate  location 
of  parts  status  by  a  variety  of  categories  which  should  enhance  MICAP 
verification  in  the  future. 

SECTION  THREE  -  REPAIR  CYCLE  TIME  EVALUATION 

HYPOTHESIS:  Aggregate  Repair  Cycle  Time  (RCT)  for  workcenters  operating  under 
Increment  VI  should  be  reduced  if  a  real  increase  in  production  processing 
efficiency  has  occurred. 

METHODOLOGY:  Dover  AFB  RCT  data  was  analyzed  for  606  stock  items  associated 

with  six  shops  which  used  Increment  VI.  We  also  analyzed  263  items  associated 
with  shops  which  did  not  use  Increment  VI.  This  was  done  to  isolate  outside 
influences  which  may  have  caused  changes  in  average  RCT.  Increment  VI 

"before"  data  consisted  of  the  second  calendar  quarter  of  1982  through  the 
tirst  calendar  quarter  of  1983,  while  "after”  data  consisted  of  the  second 
calendar  quarter  of  1983  through  the  first  calendar  quarter  of  1984.  This 
test  was  also  applied  to  average  NRTS/Condemned  days  over  the  same  period. 

ANALYSIS:  The  analysis  was  performed  by  computing  an  average  RCT  value  for 

each  stock  item  for  the  twelve  month  period  prior  to  Increment  VI 

implementation  and  for  the  first  twelve  month  period  after  implementation. 

The  difference  in  the  average  value  was  then  tested  for  statistical 
significance  using  a  matched-pair  t-test.  This  test  was  also  applied  to  NRTS 
days  over  the  same  period. 


ANALYSIS:  Audit  trails  were  established  for  items  in  the  repair  cycle 
process.  MDC  inputs  were  checked  after  the  items  were  returned  to  RPC,  prior 
to  their  return  to  supply.  No  errors  were  detected  in  the  parts  sample, 
attempts  to  "fool"  the  system  with  erroneous  serial  numbers  and  Work  Unit  Code 
inputs  were  rejected  in  100  percent  of  samples.  Parts  could  not  be  closed  out 
of  the  system  until  MDC  had  been  input.  The  program  also  checked  to  see  if 
bits  and  pieces  were  still  on  order  and  would  not  allow  job  closure  until 
repair  parts  were  either  received  or  cancelled. 

FINDING:  Increased  accuracy  of  MDC  and  Supply  data  was  achieved.  The  edits 

included  in  Increment  VI  represent  a  major  step  forward  from  independent 
supply  and  maintenance  data  systems.  From  a  maintenance  standpoint,  Increment 
VI  requires  maintenance  to  input  accurate  MDC  and  combines  this  with  edits  on 
parts  on  order.  Thus,  unneeded  parts  must  be  cancelled  prior  to  work  order 
closeout.  Although  the  requirement  to  input  AWP  data  has  created  an 
additional  workload  (see  Table  1,  subprocess  C),  in  our  opinion,  the  potential 
for  dollar  savings  and  increased  data  accuracy  far  outweigh  the  additional 
time  required. 

bENEF’iT  4:  Increase  MDC  reporting  percentage  rate. 

ANALYSIS:  We  collected  MDC  off-equipment  units  completed  by  Avionics  shops 
from  the  four  calendar  quarters  prior  to  Increment  VI  implementation,  and  the 
five  quarters  following  its  implementation.  As  an  independent  verification 
source,  we  also  collected  Supply  units  returned-to-service  (RTS)  for  the  same 
shops  over  this  period.  We  found  that  MDC  off-equipment  units  completed  were 
steadily  increasing  from  about  3000  units  to  over  6000  units  while  the  number 
of  RTS  units  showed  a  faiCly  constant  trend.  The  table  below  shows  MDC  off- 
equipment  units  completed  versus  RTS  units.  If  every  unit  returned-to-service 
was  reported  in  MDC,  off-equipment  units  completed  should  equal  at  least  100 
percent  of  RTS  units.  This  was  clearly  not  the  case  prior  to  Increment  VI 
implementation  (quarters  82-1  through  82-4).  After  Increment  VI  implementa¬ 
tion  began  in  quarter  83-1,  MDC  off-equipment  units  more  than  doubled  and 
exceeded  100%  of  RTS  units.  This  indicates  that  units  being  returned  to 
service  in  Supply  are  now  being  documented  in  MDC.  The  MDC  units  complete 
quantity  also  includes  such  maintenance  actions  as  Condemned,  Not-Reparable- 
'l'his-Station,  etc. 

Table  2.  MDC  Off-Equipment  Units  Vs  RTS  Units 


Cal.  0tr« 

82-1 

82-2 

82-3 

82-4 

83-1 

83-2 

83-3 

83-4 

84-1 

MDC  Units 

KTS  Units 

3183 

3018 

2663 

3087 

3974 

3288 

4226 

2976 

6758 

3173 

6058 

2280 

6218 

2525 

MDC  X  of  RTS 

105% 

86% 

72% 

89% 

120% 

142% 

212% 

265% 

246% 

FINDING:  MDC  off-equipment  units  reported  has  shown  a  more  than  100  percent 
increase  since  Increment  VI  implementation.  Comparison  of  MDC  and  Supply  data 
indicates  that  RTS  units  are  now  being  fully  reported  in  MDC. 


an  average  of  28  seconds.  FMS  schedulers  took.  1  minute,  7  seconds  per  part 
without  Increment  VI  or  an  advantage  of  39  seconds  for  Increment  VI  users. 

For  in-shop  processing,  (see  Appendix  A,  worksheet  1),  subprocesses  b,  C, 
and  L>,  AMS  shops  (with  Increment  VI)  took  an  average  of  9  minutes,  30  seconds 
to  in-process  each  part  while  FMS  shops  (without  Increment  VI)  took  11 
minutes,  21  seconds. 

Ordering  bits  and  pieces  took  5  minutes,  52  seconds  in  AMS  shops  while  FMS 
shops  required  an  average  of  4  minutes,  8  seconds  (it  took  AMS  shops  longer  to 
till  in  AWP  data  for  the  automated  AFTO  350).  Processing  NKTS  items  back  to 
KPC  took  6  minutes,  26  seconds  in  AMS  and  8  minutes,  5  seconds  in  FMS. 

when  processing  parts  for  return  to  Supply,  AMS  RPC  schedulers  (with 
increment  VI)  took  13  seconds  to  process  a  part  for  return  to  supply  while  FMS 
schedulers  took  39  seconds  (see  Appendix  A,  worksheet  1,  subprocess  K).  This 
2b  second  advantage  for  increment  VI  users  resulted  from  the  automation  of  the 
AFTO  350  tag. 

Avionics  production  schedulers  reported  no  appreciable  change  in  DIFM 
reconciliation  time.  In  place  of  the  visible  files,  schedulers  were  using  the 
AMS /UPS  M376S14  printout  and  the  Shop  Repair  Cycle  Assets  Report  to  validate 
the  k-26  DIFM  Listing  from  Supply.  R-26  updates  were  still  accomplished  using 
the  handscrlbed  AF  Form  1998.  An  attempt  had  been  made  to  update  the  R-26 
through  a  card  interface  to  the  Supply  computer  but  was  found  to  produce 
erroneous  data  and  was  discontinued. 

Table  1.  Increment  VI  vs  Manual  Processing  Time  (Minutes) 


► 

Average  Time 

INC  VI 

£gl 

bubprocess 

AMS 

FMS 

D1FF 

S 

A 

Processing  parts  from  KPC  to  the  shops. 

0:28 

1:07 

39 

b 

Processing  parts  for  repair  in  the  shop. 

9:30 

11:21 

-1:51 

C 

Ordering  repair  parts. 

5:52 

4:08 

+1:44 

D 

Close  out  NRTS  action  in  shop. 

6:26 

8:05 

-1:39 

Processing  parts  from  KPC  to  Suppl 


TOTAL 


22:29  25:20  -2:51 


FINDING:  Increment  VI  produced  an  overall  net  advantage  of  2  minutes  51 

seconds  per  part  processed.  Manual  documentation  was  reduced,  but  DIFM 
reconciliation  time  was  not  significantly  affected. 

nhNLFIT  3:  Increase  accuracy  of  MDC  and  supply  data. 
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and  found  that  AMS/GPs  status  was  almost  always  one  day  ahead  of  the  supply 
computer  status  and  in  some  cases,  two  days  ahead. 

BINDING:  PARTIALLY  MET.  The  data  needed  to  interface  the  two  systems  is 
available.  The  disparities  of  feeding  data  from  a  "batch-oriented"  scheduling 
system  through  a  "real-time"  maintenance  information  system  to  a  “batch 
processed"  supply  information  system  has  led  to  problems  in  the  comparability 
of  maintenance  data  versus  supply  data.  Detailed  processing  procedures  must 
oe  developed  to  insure  data  timeliness  before  the  two  systems  are  interfaced. 


SECTION  TWO  -  POSTULATED  BENEFITS: 

Benefits  1,  2,  7  and  8  are  very  closely  related.  The  activities 

associated  with  these  benefits  often  overlap.  Rather  than  attempt  to  assign 
each  activity  to  an  associated  benefit,  we  combined  our  analysis  to  cover  all 
four  of  these  postulated  benefits. 

BENEFIT  1:  Reduce  the  time  required  to  schedule  and  monitor  status  of  in-work 
(INW)  reparable  assets. 

BENEFIT  2:  Reduce  the  time  required  to  process  components  into  the  repair 
cycle  process. 

BENEFIT  7:  Reduce  the  time  required  for  due-in-f rom-maintenance  DIFM  listing 
reconciliation. 

BENEFIT  8:  Reduce  manual  documentation  required  to  track  the  flow  of 
reparables. 

ANALYSIS:  The  above  benefits  encompass  the  administrative  activities  involved 
in  getting  a  part  through  the  entire  repair  cycle  process.  This  includes 
initial  RPC  processing,  shop  processing,  and  RPC  processing  for  return  to 
Supply.  We  divided  the  overall  process  into  several  sub-processes  and  broke 
these  down  by  Individual  activities  as  shown  in  Appendix  A,  Worksheet  1.  We 
tracked  parts  through  the  repair  cycle  process  both  in  FMS  (without  Increment 
VI)  and  in  AMS  (with  Increment  VI)  to  see  if  the  Increment  VI  was  more 
efficient.  We  looked  at  the  manual  versus  automated  documents  used  to 
accomplish  the  activities,  along  with  the  average  time  required  to  perform  the 
activity  with  and  without  Increment  VI. 

in  some  cases,  we  found  that  Increment  VI  users  were  still  tracking  repair 
cycle  information  in  shop  logs  even  though  the  information  was  available 
on-line  in  the  computer.  This  appeared  to  be  a  back-up  measure  in  case  the 
computer  went  down;  however,  the  same  information  could  be  kept  by  simply 
printing  out  the  shop  report  9129.  Keeping  manual  logs  was  a  local  management 
decision  which  does  not  take  full  advantage  of  Increment  VI.  Therefore,  to 
provide  a  more  accurate  picture  of  the  repair  cycle  process  using  Increment 
VI,  manual  log  entries  accomplished  by  AMS  workcenters  were  not  included  in 
the  analysis. 

For  Initial  RFC  processing,  (see  Appendix  A,  worksheet  1),  subprocess  A, 
Schedulers  using  Increment  VI  in  the  AMS  RPC  processed  parts  to  the  shops  in 
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ANALYSIS:  Job  averages  (or  more  properly,  "job  standards")  were  present  in 

the  system.  Shop  chiefs  and  schedulers  alike  expressed  a  high  degree  of 
confidence  in  these  standards  and  said  they  were  being  used  to  establish  KTJCs 
and  to  compute  shop  backlogs. 

FlNUiNG:  FULLY  MKT. 

UbJKGTIVK  9:  Provide  on-line  input/edit  of  repair  cycle  data. 

ANALYSIS:  User's  demonstrated  the  ability  to  inquire/edit  the  full  range  of 
repair  cycle  data.  This  capability  provides  a  solid  basis  for  development  of 
a  ma i ntenance/supply  interface.  RPC  schedulers  and  shop  personnel  demon¬ 
strated  the  on-line  input  and  editing  of  repair  cycle  data  for  individual 
assets.  Users  expressed  a  high  degree  of  confidence  and  satisfaction  with 
this  capability. 

F INUlNU:  FULLY  MKT. 

ObJKCTIVK  10:  Provide  automated  AFTO  Form  349  maintenance  data  collection 
(.MUG)  records  for  repair/calibration  actions. 

ANALYSIS:  Automated  M1)C  had  fully  replaced  manual  MUC  in  the  Avionics 
Maintenance  Squadron.  Program  edits  prevented  a  wide  variety  of  common  MUC 
errors.  Users  were  the  strongest  supporters  of  automated  MUC  because  they  can 
now  get  "do  it  yourself"  on-line  MUC  feedback.  Avionics  RPC,  PMK  schedulers 
and  shop  personnel  demonstrated  this  capability  along  with  input  edits  which 
prevented  most  types  of  common  errors. 

FiNUiNC:  FULLY  MKT. 

ubJhCTiVK  11:  Provide  automated  AFTU  Form  350  reparable  item  processing 
Lags  tor  repair  actions. 

ANALYSIS:  This  capability  was  fully  provided  and  was  being  used  to  process 

all  reparables  in  the  Avionics  Maintenance  Squadron.  Users  demonstrated  the 
capability  ol  inputting  and  printing  AFTO  Form  350  item  processing  tags  for 
repa i r  act  ions . 

FINDING:  FULLY  MKT. 

ObJKCTIVK  12:  Provide  for  an  interface  with  AMS  increment  VII  to  provide 
real-time  tracking  of  parts  from  requisition  to  time  of  turn  in. 

ANALYSIS:  increment  VI  lias  established  the  basis  for  the  maintenance/supply 

interface.  The  specified  data  elements  for  both  maintenance  and  supply 
requirements  to  be  interfaced  are  present.  However,  Increment  VI  was  designed 
to  operate  in  "real  time"  while  supply  repair  cycle  data  is  "batch-processed." 
During,  peak  usage  periods,  batch  processing  of  AMS/CPS  data  caused  delays  in 
parts  processing  for  turn  in.  In  many  cases,  repaired  parts  were  not 
documented  by  supply  closeout  time,  and  consequently  were  not  processed 
through  supply  until  the  following  day.  because  of  this  anomaly,  a  previous 
attempt  to  update  supply  repair  cycle  data  through  a  card  interlace  proved 
unworkable.  Also,  we  compared  sarr  e  R-2b  reports  to  AMS/GPS  off-line  reports 


Subprocess  1)  -  Continued 

Activities 

Average  Time 

7  AMS 

No  action  required 

00  sec 

7  FMS 

Annotate  R-26  report 

05  sec 

Total 

process  time  -  AMS 

6  min  26  sec 

Total 

process  time  -  FMS 

8  min  05  sec 

KPC  PROCESSING  PARTS  FOR  RETURN  TO  SUPPLY 


Subprocess  E  -  Processing  parts  from  RPC  back  to  Supply. 

Activities _ Average  Time 

1  AMS  Inspect  documentation  attached  to  part  03  sec 

1  FMS  Inspect  documentation  attached  to  part  03  sec 


/  AMS  Annotate  Automated  AFTO  Form  35U  08  sec 

2  FMS  Annotate  AFTO  Form  350  30  sec 


3  AMS  Annotate  R-26  02  sec 

3  FMS  Annotate  R-26  02  sec 


A  AMS  No  Action  kequired  00  sec 

4  FMS  Annotate  RPC  log  04  sec 


Total  process  time  -  AMS  (Inc  VI  RPC)  13  sec 

Total  process  time  -  FMS  (Non-Inc  VI  RPC)  39  sec 


Note:  In  cases  where  processes  were  exactly  the  same  for  both  Increment  VI 

shops  and  non-increment  VI  shops,  the  overall  average  time  was  used. 


WORKSHEET  2 


a.  Evaluation  Objective:  Determine  if  AMS  Increment  VI  has  reduced  time 
required  to  process  and  monitor  reparable  assets  from  the  RPC  viewpoint. 
Determine  if  workload  has  been  increased,  decreased  or  unchanged.  Determine 
it  users  perceive  and  increase,  decrease,  or  no  change  in  their  efficiency  as 
a  result  of  Increment  VI  implementation. 

b.  Methodology:  Interview  RPC  and  PMEL  personnel  to  determine  their 
perceptions  of  Increment  VI  impacts  on  their  jobs. 


Organization:  Avionics  Maintenance  RPC  and  PMEL  RPC 


(1)  Has  AMS  Increment  VI  changed  your  processing  time  for  assets? 

AMS  RPC  schedulers  -  It  has  decreased  processing  time  by  about  half 
primarily  due  to  automation  of  the  AFTO  Form  350  and  the  elimination  of  the 
scheduling  boards. 

AMS  AWP  monitor  -  It  has  increased  processing  time  for  AWP  parts  due  to 
the  requirement  to  enter  supply  data  into  the  maintenance  computer.  If  the 
maintenance  and  supply  computers  could  talk  to  each  other  this  problem  would 
be  eliminated. 

PMEL  scheduler  -  Automated  MDC  has  decreased  processing  time  for  both 
calibration  items  and  reparables. 

(2)  Has  Increment  VI  added  to  your  paperwork? 

AMS  RPC  schedulers  -  It  has  cut  paperwork. 
aMS  AWP  monitor  -  It  has  increased  paperwork. 

PMEL  scheduler  -  It  has  cut  paperwork  greatly. 

(3)  What  additional  forms  are  now  required? 
aMS  RPC  schedulers  -  None. 

AMS  AWP  monitor  -  None,  but  the  amount  of  information  tracked  on  the  AWP 
visible  file  has  increased. 

PMEL  Scheduler  -  None 

(4)  What  new  reports  must  you  now  fill  out  or  monitor? 

AMS  RPC  schedulers  -  The  on-line  9129  (shop  DIFM  report)  and  the  off-line 
VIRP-M 1/6S14  (Shop  Repair  Cycle  Assets  Report)  replaced  the  old  visible  files 
on  the  scheduling  board. 


aMS  AWP  monitor  -  The  off-line  VIRP  M376S17  (AMS  supply  listing  for  AWP 
section)  must  now  be  compared  to  the  D-19  (AWP  validation  listing)  from 
supply . 

ttthL  scheduler  -  The  AMS/GPS  master  equipment  list  replaces  the  old  master 
1 U  listing  from  the  B-3500. 

(5)  How  much  added  work  time  is  spent  using  these  reports. 

aMS  RPC  schedulers  -  About  the  same  amount  of  time  is  now  spent  using 
these  reports  as  was  previously  spent  using  the  visible  file. 

AMS  AWP  monitor  -  It  now  takes  1  to  2  hours  longer  to  complete  AWP 
validation  because  AWP  information  now  has  to  be  updated  in  both  the 
maintenance  and  supply  computers. 

PMEL  scheduler  -  The  AMS/GPS  report  has  resulted  in  a  net  savings  of  time 
since  it  is  automatically  updated  by  MDC.  The  old  B-35U0  master  ID  listing 
was  always  outdated  and  a  lot  of  time  and  trouble  was  spent  getting  it 
updated. 

(6)  What  forms  are  you  no  longer  using  as  a  result  of  Increment  VI?  For  each 
lorm  no  longer  used  (if  any): 

AMS  RPC  schedulers  -  The  AFTO  350  and  the  AF  2443  have  now  been  automated. 
AMS  AWP  Monitor  -  N/A. 

PM EL  Scheduler  -  AFTO  350  is  now  automated. 

(7)  How  long  does  it  take  to  fill  out  this  form? 

AMS  RPC  schedulers  -  About  5  minutes  to  key  in  the  entire  AFTO  350.  The 
aF  2443  is  automatically  completed  by  the  computer.  (Note:  During  stopwatch 
checks,  schedulers  routinely  input  AFTO  350  scheduling  data  in  an  average  of 
13  seconds.) 

AMS  AWP  Monitor  -  N/A 

PMhl,  Scheduler  -  Just  a  few  seconds  to  input  the  AFTO  350.  Before 
increment  VI  it  took  a  couple  of  minutes. 

(d)  Has  Increment  VI  changed  the  number  of  phone  calls  and/or  visits  you  used 
to  make  to  the  shops? 

AMS  RPC  Schedulers  -  Phone  calls  increased  primarily  because  parts  cannot 
be  closed  out  in  the  computer  until  MDC  is  completed.  Shops  often  get  behind 
in  MDC  inputs  and  must  be  reminded. 

AMS  AWP  Monitor  -  Phone  calls  and  visits  increased  because  parts  cannot  be 
closed  out  in  the  computer  with  bits  and  pieces  on  order  in  supply.  If  the 
supply  computer  could  interface  with  maintenance  computer  when  a  part  is 
issued,  there  would  be  fewer  phone  calls  required. 
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HIEL  Scheduler  -  Dramatically  decreased.  AMS/GPS  Management  notices  have 
cut  the  number  of  items  going  overdue  calibration,  reducing  the  number  of 
times  the  shop  must  be  called. 

(9)  What  automated  reports  did  you  use  before  Increment  VI  and  what  automated 
reports  are  you  using  now? 

AMS  RPC  Schedulers  -  Mainly  the  R-26  prior  to  Increment  VI.  Now  the  9129 
report  and  the  VIRP  M376S14  have  replaced  the  scheduling  boards  and  visible 
tiles  while  the  R-2b  is  still  in  use. 

aMS  AWP  Monitor  -  The  D19  prior  to  Increment  VI.  Now  the  VIRP  M376S17  has 
been  added. 

PMEL  Scheduler  -  The  master  ID  listing  from  the  B-3500  before  Increment 
VI.  This  was  replaced  by  the  AMS/GPS  master  equipment  listing,  a  much  better 
product . 

(10)  are  you  confident  that  repair  cycle  asset  (or  calibration)  status  in 
AMS/GPS  is  up  to  date  and  accurate. 

AMS  RPC  Schedulers  -  As  long  as  the  computer  does  not  go  down  for  more 
than  a  couple  of  hours.  However  there  is  little  confidence  in  mockup  status 
^serviceable  or  unserviceable)  in  the  9129  report  and  ETJCs  are  not  updated  by 
shops  in  a  timely  manner. 

AMS  AWP  Monitor  -  The  supply  computer  is  more  up  to  date.  It  takes  a  lot 
of  work,  to  update  AF  2005  information  in  the  maintenance  computer. 

PMEL  Scheduler  -  Much  more  confident  in  calibration  status  information. 

(11)  How  do  the  supply  R-26  report  and  AMS/GPS  status  listings  compare  for 
accuracy  and  timeliness? 

aMS  RPC  Schedulers  -  The  R-26  at  the  scheduler's  board  is  kept  updated 
continuously  (manually).  It  is  the  most  accurate  document. 

aMS  AWP  Monitor  -  The  R-26  is  more  accurate  and  timely. 

PMEL  Scheduler  -  N/A. 

(12)  Are  you  required  to  reconcile  supply  listings  with  AMS/GPS  listings. 

aMS  RPC  Schedulers  -  Yes,  but  it  requires  no  more  time  than  it  did  with 
visible  files.  Maybe  less. 

AMS  AWP  Monitor  -  Yes  and  it  takes  an  additional  1  to  2  hours  per  day. 

PMEL  Scheduler  -  N/A 

(13)  Have  shop  AWM,  AWP,  or  calibration  backlogs  been  reduced  as  a  result  of 
Increment  VI? 
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AMS  RPC  Schedulers  -  AWM  has  shown  as  increase  on  the  automated  AF  2443. 
This  is  because  of  machine  downtime  and  the  shops  not  updating  status  in  a 
t ime ly  ma  nne  r . 

AMS  AWP  Monitor  -  AWP  time  has  increased  because  the  requirement  to  enter 
AF  2005  information  in  the  maintenance  computer  has  caused  an  increase  in 
processing  time. 

PMEL  Scheduler  -  Backlogs  have  decreased. 

(14)  Regarding  reparable  asset  information: 

(a)  What  information  should  be  deleted  from  AMS/GPS? 

AMS  RPC  schedulers  -  None. 

AMS  AWP  Monitor  -  There  is  no  need  for  tech  order,  figure,  index,  and  time 
ordered  in  the  9131  input. 

PMEL  Scheduler  -  None. 

(b)  What  information  should  be  added  to  AMS/GPS? 

AMS  RPC  Schedulers  -  An  automatic  ETJC  management  notice  would  be  very 
helpful.  The  computer  should  automatically  show  all  ETJC's  due  in  the  next 
hour . 

AMS  AWP  Monitor  -  It  needs  to  be  able  to  communicate  with  the  supply 
computer  to  save  inputting  the  same  information  into  two  different  computers. 

PMEL  Scheduler  -  Job  standards  for  item  calibration  should  be  printed  on 
the  AFTO  Form  350. 

(c )  What  changes  should  be  made  to  screen  input/output  formats? 

aMS  RPC  Schedulers  -  There  should  be  a  way  to  terminate  the  9129  report 
and  to  back  up  to  previous  pages.  Currently,  you  have  to  keep  pressing  PA1 
until  you  get  to  the  end  of  the  report  and  some  shops  have  20  to  30  page 
reports.  (Note:  This  problem  has  now  been  corrected.) 

AMS  AWP  Monitor  -  The  9131  input  screen  needs  to  be  easier  to  follow  and 
should  not  require  so  many  inputs.  Better  yet,  it  could  be  eliminated 
altogether  if  the  supply  computer  could  send  the  maintenance  computer  AF  Form 
2005  information. 

PMEL  Scheduler  -  The  discrepancy  block  on  the  AFTO  Form  350  needs  to  be 
longer . 

(d)  What  changes  should  be  made  to  print  out  formats? 

AMS  RPC  Schedulers  -  The  AFTO  Form  350  printed  copy  gets  greasy,  wadded  up 
and  unusable.  Having  to  get  a  new  copy  printed  out  is  an  irritant  and  you 
cannot  get  a  new  copy  when  the  computer  is  down.  The  AFTO  Form  350  should  be 
printed  on  cardboard. 


AMS  AWP  Monitor  -  None. 

PMEL  Scheduler  -  None. 

(15)  Is  Increment  VI  response  time  adequate  for  your  needs? 

AMS  RPC  Schedulers  -  Yes  except  around  1330  to  1530  the  shops  bog  down  the 
system. 

AMS  AWP  Monitor  -  Yes. 

PMEL  Scheduler  -  Yes  except  an  hour  or  so  before  the  day  shift  gets  off, 
response  time  gets  pretty  slow. 


WORKSHEET  3 


a.  Evaluation  Objective:  Determine  if  AMS  Increment  VI  has  eliminated 

the  time  required  to  compute  shop  backlogs. 

b.  Methodology:  Through  interviews  with  RPC  schedulers  in  AMS  and  KMS, 
define  the  individual  steps  involved  in  gathering  backlog  information  and 
computing  shop  backlogs.  Through  stopwatch  studies,  quantify  the  time 
required  to  perform  shop  backlog  computations  manually. 


Process  -  Computing  shop  backlogs. 

Description  of  Activity 

1.  Count  AWM  INW  units  each  shop  from  visible  file. 


2.  Count  AWM  INW  hours  each  shop  from  visible  file. 


Total  time  to  perform  in  FMS  32  min  30  sec 

Total  time  to  perform  in  AMS  (automatically  computed)  0  min  08  sec 


Note:  In  FMS,  these  activities  were  not  performed  in  finite  sequence,  so  we 

measured  the  overall  time  to  compute  the  backlog.  In  AMS,  the  scheduler 
requested  the  shop  backlog  report  from  the  AMS/CPS  terminal  and  the  automated 
aF  2443  was  displayed  In  b  seconds. 
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APPENDIX  B 


HUMAN  FACTORS  ASSESSMENT 


AIR  FORCE  HUMAN  RESOURCES  LABORATORY 
WRIGHT-PATTERSON  AFB,  OHIO 
MAY  1984 


INTRODUCTION : 


The  AMS  Test  Program  is  designed  to  test  the  impact  and  usefulness  of 
automating  aircraft  maintenance  management  processes  using  the  database  of  the 
c-5  Malfunction  Detection  Analysis  and  Recording  System  (MADARS)  and  the 
associated  Ground  Processing  System  (GPS).  This  program  provides  an 
opportunity  to  exploit  computer  technology  to  assist  management  in  a  direct 
mission  support  maintenance  environment.  The  program  is  being  implemented  in 
seven  increments,  and  is  being  evaluated  in  an  operational  base  environment  at 
Dover  APB,  Delaware.  The  Air  Force  Logistics  Management  Center  (AFLMC)  has 
overall  program  evaluation  responsibility,  with  the  Air  Force  Human  Resources 
Laboratory  (AFHKL)  participation  limited  to  evaluation  of  user  acceptance  and 
user  perception  of  usability  of  the  system.  The  acceptance  and  usability 
evaluation  for  Increment  VI,  Maintenance  Production  Scheduling  Process  was 
conducted  during  the  period  from  14  through  17  November  1983.  A  follow-on 
evaluation  was  conducted  on  9  and  10  May  l!r84.  This  report  is  submitted  to 
summarize  the  findings  of  the  follow-on  evaluation  of  Increment  VI. 

The  overall  program  is  designed  to  increase  the  effectiveness  of 
maintenance  management  personnel  by  providing  real-time  updating  of 
maintenance  information,  computer  generation  of  aircraft  maintenance  forms, 
and  elimination  of  the  requirement  for  certain  manual  files.  Increment  VI, 
Maintenance  Production  Scheduling  Process  is  responsible  for  the  management 
and  scheduling  of  two  areas:  repair  cycle  assets  and  precision  measurement 

equipment.  The  repair  cycle  asset  control  system  provides  for  processing, 
accountability  and  control  of  repair  cycle  items.  The  precision  measurement 
equipment  scheduling  function  manages  and  schedules  calibration  and  repair  of 
requirement  precision  measurement  equipment. 

P  Li  HP  US  hi : 

The  purpose  of  this  study  was  to  reevaluate  the  attitudes  of  maintenance 
personnel  toward  the  changes  incorporated  by  Increment  VI  of  the  AMS  Test 

Program  . 

Mi  !  MOD: 

1  flies t  ionna ires  and  interviews  were  used  to  collect  data  from  personnel  in 
in*  a  Sbt  h  MAW  who  are  primary  users  of  the  new  processes.  A  total  of  30 
jin- ,  t  i  onna  i  res  were  distributed  to  various  individuals.  All  individuals  were 
> hus'ui  randomly  throughout  the  using  workcenters. 

1.  I  raining  -  Surveys  showed  a  drastic  change  from  the  November  evaluation  in 
requirements  (reference  question  V-F).  In  November,  22%  of  the  surveys  stated 
that  lack  of  training  was  one  of  the  largest  problems  with  Increment  VI.  This 
was  only  reflected  on  b%  of  the  May  surveys.  Responses  to  questions  in 
Section  11  Indicate  relatively  little  training  is  required  to  understand  and 
use  the  system.  This  indicates  it  is  an  extremely  user-friendly  system. 

l.  Difficulty  in  Implementation  -  Responses  to  all  items  in  Section  III  of 
the  questionnaire  indicate  tasks  affected  by  the  Increment  VI  are  easier  to  do 
with  the  new  system  than  with  the  old  method.  There  is  a  noticeable 


difference  in  responses  to  the  November  and  May  surveys.  Responses  to  the 
November  survey  indicated  only  5  of  the  7  tasks  are  easier  with  the  new 
system.  The  May  survey  indicated  that  all  tasks  are  easier.  Also,  the 
percentages  of  positive  responses  to  each  question  were  from  10  to  20  percent 
higher  than  for  the  May  survey.  These  results  indicate  as  they  gain 
experience  with  the  system,  more  technicians  find  the  task  easier  to  perform 
using  the  new  system  than  using  the  old  method. 

3.  Usability  -  The  usability  and  user  friendliness  of  the  system  were  highly 
praised  in  the  most  recent  interviews.  These  comments  are  generally  supported 
by  responses  to  Section  IV  of  the  questionnaire.  Again,  there  was  a  definite 
trend  toward  more  positive  response  to  the  survey  questions  in  May  than  in 
November.  For  example,  63  percent  of  the  responses  to  question  IV-A  indicated 
the  new  method  is  less  difficult  to  learn  than  the  old  method.  In  November, 
only  17  percent  indicated  the  new  method  is  easier. 

4.  User  Acceptance  -  Responses  in  this  area  indicate  the  system  is  well 
accepted  by  the  users  and  it  provides  them  with  the  information  they  need  to 
do  their  jobs.  Most  (63  percent)  felt  that  Increment  VI  should  be  permanently 
added  to  the  Ground  Processing  System.  Again,  the  responses  in  May  were 
noticeably  more  positive  than  the  responses  in  November. 

UlSCUSSiON: 

This  study  has  provided  information  on  acceptance  and  usability  of  Increment 
VL  of  the  AMS  Test  Program.  The  evaluation  is  based  strictly  on  the  results 
of  questionnaires  and  on  personal  interviews.  Attachment  1  shows  both  the 
results  of  the  November  evaluation  and  the  May  reevaluation.  There  was  a 
noticeable  increase  in  the  positive  attitudes  toward  Increment  VI  after  it  had 
had  time  to  grow  and  mature.  Personnel  were  better  utilizing  the  available 
products  and  the  efficiency  of  the  system  has  improved  considerably.  The 
overall  feeling  expressed  by  the  users;  it  is  getting  better  and  better  with 
time . 

Kfc.COM  MhNDAT IONS: 

1.  Continue  to  observe  the  effects  of  AMS  on  maintenance  efficiency  and 
sortie  production. 

2.  Implement  Increment  VII  as  soon  as  possible  and  develop  more  uses  of 
the  system. 

3.  Accept  user  Inputs  to  improve  the  system. 

4.  Try  to  incorporate  as  many  aspects  of  AMS  as  possible  into  the  Core 
Automated  Maintenance  System  (CAMS). 


Attachment  1 


GROUND  PROCESSING  SYSTEM  (GPS)  INCREMENT  VI 
IMPLEMENTATION  QUESTIONNAIRE 

The  Ground  Processing  System  (GPS)  has  been  modified  to  provide  greater 
maintenance  management  processes  through  automation.  These  processes  are  to 
be  accomplished  in  seven  increments.  Thus  far,  increments  one  through  six 
have  been  implemented. 

The  Air  Force  Human  Resources  Laboratory  has  been  requested  to  determine 
user  perceptions  of  and  acceptance  of  the  new  processes  and  products  which 
resulted  from  implementing  the  new  system. 

This  questionnaire  has  been  designed  specifically  to  ascertain  your  views 
on  Increment  VI,  Maintenance  Production  Scheduling  Process. 

Your  response  to  this  questionnaire  will  aid  the  Laboratory  in  determining 
user  perception  of  system  usability,  user  acceptance  and  difficulties 
encountered  during  system  implementation.  In  addition,  any  comments  you  may 
have  concerning  the  acceptance,  usability  or  implementation  of  the  entire  GPS 
program  (Increments  I-VI)  not  covered  herein  can  be  stated  on  the  back  of  this 
questionnaire.  All  responses  to  this  questionnaire  will  be  kept  in  strict 
confidence. 

I.  POPULATION  PROFILE 

a.  What  is  your  job  title? 
tt.  What  is  your  AFSG? 

G.  How  long  have  you  been  at  your  present  job? 

INDICATE  YOUR  RESPONSE  TO  THE  REMAINING  QUESTIONS  BY  MARKING  AN  "X"  NEXT  TO 
THE  APPROPRIATE  ANSWER. 

II.  TRAINING 

a.  How  much  formal  (classroom)  training  on  Increment  VI  did  you  receive 
prior  to  implementation? 


May 

84 

Nov  83 

70% 

(1) 

None 

87% 

7% 

(2) 

One  hour 

3% 

1  3% 

(3) 

Two  Hours 

3% 

n 

(4) 

Four  Hours 

3% 

7% 

(5) 

Six  or  More  Hours 

3% 

n.  How  much  on-the-job  training  on  Increment  VI  did  you  receive  prior  to 
i  mp 1  erne nlat ion? 


30 


)• 


May 

84 

Nov  83 

t>0% 

(1) 

None 

537. 

77. 

(2) 

One  Hour 

27% 

17% 

(3) 

Two  Hours 

3% 

3% 

(4) 

Four  Hours 

3% 

1'37. 

(3) 

Six  or  More  Hours 

13% 

C.  How  much  additional  training  time  do  you  feel  would  be  required  to 
fully  understand  the  new  system?  (On  reverse  please  indicate  in  what  areas 
you  would  increase  training.) 


May 

84 

Nov  83 

577. 

(1) 

None 

33% 

107. 

(2) 

One  Hour 

10% 

20% 

(3) 

Two  Hours 

3% 

3% 

(4) 

Four  Hours 

27% 

10% 

(5) 

Six  or  More  Hours 

27% 

D.  How  much  training  time  could  be  deleted  from  the  formal  classroom 
training  you  received  on  Increment  VI  and  still  be  adequate?  (On  reverse 
please  indicate  what  areas  of  training  could  be  deleted.) 


May 

84 

Nov  83 

33% 

(1) 

None 

17% 

3% 

(2) 

One  Hour 

0% 

3% 

(3) 

Two  Hours 

3% 

3% 

(4) 

Four  Hours 

0% 

0% 

(5) 

Six  or  More  Hours 

3% 

57% 

(6) 

Did  Not  Receive  Formal  Training 

77% 

III.  DIFFICULTY  IN  IMPLEMENTATION 

During  the  implementation  phase  of  Increment  VI,  how  would  you  describe 
the  amount  of  difficulty  you  encountered  performing  the  following  tasks  when 
compared  to  the  old  method? 

A.  Monitoring  the  status  of  due-in-f rom-maintenance  (DIFM)  items: 


May 

84 

Nov  8  3 

10% 

(1) 

More  difficult 

than 

before 

23% 

3% 

(2) 

No  Change 

10% 

437. 

(3) 

Less  Difficult 

than 

before 

307. 

43% 

(4) 

No  Comment 

37% 

n.  Monitoring  the  status  of  precision  measurement  equipment  (PME): 


May 

84 

Nov  83 

3% 

(1) 

More  difficult  than 

before 

17% 

237. 

(2) 

No  Change 

17% 

437. 

(3) 

Less  Difficult  Than 

Before 

27% 

337. 

(4) 

No  Comment 

407. 

c. 

Determi ning 

the 

location  and  status 

of  an 

individual  repai 

asset : 

May 

84 

Nov  83 

107. 

(1) 

More  difficult  than 

before 

30% 

207. 

(2) 

No  Change 

23% 

33% 

(3) 

Less  Difficult  than 

Before 

17% 

37% 

(4) 

No  Comment 

30% 

u. 

Determining 

labor  hour  backlog  for 

of f-equipment 

work  and 

assets 

awaiting  repair/calibration: 

May 

84 

Nov  83 

17% 

(1) 

More  difficult  than 

before 

7% 

0% 

(2) 

No  Change 

27% 

307. 

(3) 

Less  Difficult  than 

Before 

23% 

33% 

(4) 

No  Comment 

43% 

t . 

Monitoring  estimated  times  of  job  completion 

(ETJC) 

for  off-eqi 

PMC  work  orders: 

May 

84 

Nov  83 

13% 

(1) 

More  difficult  than 

before 

3% 

13% 

(2) 

No  Change 

33% 

27% 

(3) 

Less  Difficult  Than 

Before 

10% 

47% 

(4) 

No  Comment 

537. 

y . 

Monitoring  mission  capability  (MICAP) 

repair 

cycle 

assets : 

May 

84 

Nov  83 

07. 

(1) 

More  difficult  than 

before 

10% 

137. 

(2) 

No  Change 

17% 

27% 

(3) 

Less  Difficult  Than 

Bef* re 

13% 

00% 

(4) 

No  Comment 

607. 

{j .  Determining  job  averages  for  component  repair/calibration: 
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May 

84 

Nov  83 

3% 

(1) 

More  difficult  than  before 

3% 

17% 

(2) 

No  Change 

27% 

30* 

(3) 

Less  Difficult  Than  Before 

13% 

30* 

(4) 

No  Comment 

57% 

IV. 

USABILITY 

HOW 

would  you  describe  your  experiences  in  learning 

the  new  processes? 

May 

84 

Nov  83 

13* 

(1) 

More  difficult  to  learn  than  the 
Previous  Methods. 

47% 

40% 

(2) 

No  Differences  in  Learning  the 
two  Processes. 

37% 

63% 

(3) 

Less  Difficult  to  Learn  Than  the 
Previous  Methods. 

17% 

b. 

How 

would  you  describe  your  experiences  in  retrieval  of  data  with  the 

new 

system? 

May 

84 

Nov  83 

17% 

(1) 

More  difficult  to  Retrieve  the 
Data  I  Want. 

30% 

33% 

(2) 

No  Difference  in  Retrieval 

Time  s . 

17% 

50% 

(3) 

Easier  to  Retrieve  the  Data. 

53% 

C . 

Do 

you 

find 

the 

language/terminology  used  in  the 

new  system  hard  to 

unde  rs  tand 

or  confusing: 

May 

84 

Nov  83 

u% 

(1) 

F requently 

3% 

27% 

(2) 

Occasionally 

3% 

40% 

(3) 

Seldom 

67% 

33% 

(4) 

Neve  r 

27% 

D. 

How 

has 

the 

new 

system  affected  the  number  of  phon< 

e  calls  between  the 

concerning 

Work 

Centers  ? 

May 

84 

Nov  83 

13% 

U) 

Increased  the  Number  of  Calls. 

13% 

37% 

(2) 

No  Change  in  the  Number  of  Calls. 

7  3% 

30% 

(3) 

Decreased  the  Number  of  Calls. 

13% 

t  . 

Do 

you 

feel 

that  the  automated  OPS  System  is  reliable  (operational) 

wnen  you  need  it? 
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May  84 


Nov  83 


63%  (1)  Yes  60% 

37%  (2)  No  40% 

F.  As  compared  with  the  previous  methods,  how  has  data  accessibility  from 
the  automated  system  affected  your  efficiency? 


n 


May  84 

40%  (1)  Increased  my  Efficiency  13% 

Significantly . 

23%  (2)  Increased  my  Efficiency  Slightly  30% 

17%  (3)  No  Change.  27% 

17%  (4)  Decreased  my  Efficiency  Slightly  13% 

3%  (5)  Decreased  my  Efficiency  17% 

Significantly 

0.  The  new  automated  GPS  systems  have  eliminated  the  need  to  maintain  an 
aFTO  Form  349  file. 

May  84  Nov  83 

33%  (1)  I  Strongly  Agree  17% 

23%  (2)  I  Agree  30% 

13%  (3)  I  am  Neutral  7% 

10%  (4)  I  Disagree  23% 

3%  (5)  I  Strongly  Disagree  13% 

17%  (6)  No  Comment  10% 

H.  The  availability  of  a  machine-resident  file  containing  DIFM  and  PME 
assets  status  has  been  of  benefit  to  me  in  my  job. 


Nov  83 


May  84 


Nov  83 


27% 

(1) 

I 

Strongly  Agree 

10% 

27% 

(2) 

I 

Agree 

30% 

13% 

(3) 

I 

am  Neutral 

20% 

3% 

(4) 

I 

Disagree 

3% 

3% 

(5) 

I 

Strongly  Disagree 

7% 

13% 

(6) 

No  Comment 

3% 

13% 

(7) 

I 

Do  Not  Use  That  Information 

27% 

i.  The  maintenance  of  a  machine-resident  file  of  standards/ job  averages 
tor  calculation  of  repair  times  and  estimated  in-commission  times  has  been  of 
benefit  to  me  in  my  job. 


May  84 


Nov  83 


202  (1)  I  Strongly  Agree  72 

202  (2)  I  Agree  72 

102  (3)  I  am  Neutral  272 

72  (4)  I  Disagree  32 

32  (5)  I  Strongly  Disagree  72 

102  (6)  No  Comment  102 

302  (7)  1  Do  Not  Use  That  Information  402 


j.  The  availability  of  determining  by  video  display  terminal  the  number 
of  personnel  engaged  in  a  specific  task  has  been  of  benefit  to  me. 


May 

84 

Nov  83 

302 

(1) 

I  Strongly  Agree 

32 

232 

(2) 

I  Agree 

102 

32 

(3) 

I  am  Neutral 

202 

132 

(4) 

I  Disagree 

102 

32 

(5) 

I  Strongly  Disagree 

72 

72 

(6) 

No  Comment 

102 

202 

(7) 

I  Do  Not  Use  That  Information 

402 

K.  having  complete 

machine-resident  job  standards 

have  helped 

establishing  Estimated  Time  In  Commission  (ETIC). 

May 

84 

Nov  83 

232 

(1) 

I  Strongly  Agree 

32 

172 

(2) 

I  Agree 

72 

432 

(3) 

I  am  Neutral 

572 

102 

(4) 

I  Disagree 

202 

72 

(5) 

I  Strongly  Disagree 

132 

in 


L.  With  this  new  system,  I  have  encountered  difficulties  in  generation  of 
work,  orders. 


May 

84 

Nov  83 

02 

(1) 

Yes,  Frequently 

132 

272 

(2) 

Yes,  Occasionally 

372 

432 

(3) 

No,  Seldom 

402 

302 

(4) 

Never 

102 

M.  With  this  new  system,  I  have  encountered  difficulties  in  monitoring 
work  orders. 
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** 

**v 

May 

84 

Nov  83 

0  % 

(1) 

Yes,  Frequently 

7% 

,  ■ 

20% 

(2) 

Yes,  Occasionally 

53% 

.  V  "■ 

43% 

(3) 

No,  Seldom 

33% 

«  * 

30% 

(4) 

Never 

7% 

N. 

As  compared 

with 

the  previous  methods,  I  have  encountered  difficulties 

when  an 

asset  has  extended  delays. 

May 

84 

Nov  83 

> 

3% 

(1) 

Yes,  Frequently 

10% 

- 

37% 

(2) 

Yes,  Occasionally 

30% 

■ 

27% 

(3) 

No,  Seldom 

53% 

33% 

(4) 

Never 

7% 

0. 

As  compared 

with 

the  previous  methods,  I  have  encountered  difficulties 

in  the 

close  out  of 

work 

orders . 

• 

May 

84 

Nov  83 

10% 

(1) 

Yes,  Frequently 

30% 

40% 

(2) 

Yes,  Occasionally 

33% 

23% 

(3) 

No,  Seldom 

30% 

27% 

(4) 

Never 

7% 

V.  USER  ACCEPTANCE 

COMMENTS  ABOUT 

YOUR 

ANSWERS  ARE  ENCOURAGED.  THESE 

MAY  BE  WRITTEN  ON  THE 

BACK  OF  THE  PAGES 

PLEASE  INDICATE  THE  LETTER  OF 

THE  QUESTION  YOU  ARE 

MW 

RESPONDING  TO  BEFORE  WRITING  YOUR  COMMENTS. 

ol 

A. 

Overall,  how  did 

Increment  VI.  Production  Scheduling  affect  your  lob? 

May 

84 

Nov  83 

13% 

(1) 

Made  it  More  Difficult 

33% 

*.  V- 

33% 

(2) 

Made  it  Easier 

20% 

• 

17% 

(3) 

Made  it  Longer 

30% 

77. 

(4) 

Made  it  Shorter 

0% 

30% 

(5) 

No  Change 

17% 

». *  * 

* . 

B. 

In  your  opinion, 

is  Increment  VI  a  success? 

• 

May 

84 

V  3 

33% 

(1) 

Yes 

57% 

20% 

(2) 

No 

23% 

]-■■'  ■ 

47% 

(3) 

No  Opinion 

20% 

m 

^  • 

Should  Increment 

VI,  Production  Scheduling  be  permanently  added  to  the 

**  .  . 

Ground 

Processing  System? 

>►  v 

L‘  .  * 

L.' 
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• 

C.  -.■-v-'l 

*.• 

May  84 


Nov  83 


83% 

(1) 

Yes 

50% 

10% 

(2) 

No 

30% 

27% 

(3) 

Don't  Know 

20% 

D.  Was  the 

information  you  required  to  do  your  job  readily  available  with 

Increment  VI? 

May 

84 

Nov  83 

70% 

(1) 

Yes 

57% 

20% 

(2) 

No 

37% 

10% 

(3) 

I  Do  Not  Require  Information 

7% 

From  System 

8 .  Do  you 

feel 

the  information  available  from  the  new 

automated 

system  is 

satisfactorily 

accurate  for  your  job? 

May 

84 

Nov  83 

67% 

(1) 

Yes 

73% 

17% 

(2) 

No 

17% 

17% 

(3) 

I  Do  Not  Require  Information 

10% 

F .  What  do  you 

feel 

is  the  largest  problem  with  Increment  VI? 

(Indicate 

as  many  as  you 

feel 

are  applicable.) 

May 

84 

Nov  83 

7% 

(1) 

Errors  in  Data 

12% 

10% 

(2) 

Availability  of  System  Hardware 

15% 

(Terminal  Available  to  Work,  on) 

33% 

(3) 

Computer  Down-time 

25% 

0.  Did  you  have  any  trouble  learning  to  use  GPS, 

specifically  those 

processes  associated  with 

i  Increment  VI,  in  order  to  obtain 

i  the  data 

necessary 

to  perform  your  job? 

May 

84 

Nov  83 

10% 

(1) 

Yes,  a  Lot 

0% 

30% 

(2) 

Yes,  Some 

40% 

60% 

(3) 

No 

60% 

h.  Would  you  recommend  that  Increment  VI  be  incorporated  throughout  the 
Air  Force? 
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B 
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